SAS protocol specifies a protocol stack that provides serial physical interconnect that can be used to connect storage devices such as Hard Disk Drives (HDD), host devices together. It specifies the transport layer protocols to transport SCSI commands, Serial Advanced Technology Attachment (alternatively Serial ATA or simply SATA) commands and management commands among storage devices. The SATA protocol is used in conjunction with SCSI and ATA command sets. The SAS protocol defines the function of SAS expanders, which are part of the service delivery subsystem and facilitates communication between SAS devices. In the general sense, the SAS expanders provide the switching and routing function among the SAS devices that are attached to such SAS expanders.
Multiple SAS end devices and SAS expander devices can be connected together to form a SAS topology. There can be one or multiple physical links connecting each pair of the neighbouring devices. When there is a single physical link between two devices, the associated phy on the SAS device is called a narrow port. When there are multiple physical links connecting two devices, the associated phys on an expander are considered to be a single wide port. In other words, all links of a wide port are considered to be a common logical link from routing perspective, although it allows multiple simultaneous connections to pass through the wide port at the same time. The SAS standard specifies that a legal SAS topology cannot contain a loop—i.e. there can be only one logical path to traverse the topology from any device to any other device (considering all links of a wide port to be a single link). Under such constraints, the only type of topology allowed by SAS specification is a spanning tree topology.
SAS expanders are required by the relevant American National Standards Institute (ANSI) specification to periodically generate the NOTIFY (ENABLE SPINUP) primitive that controls the spin-up of SAS drives. NOTIFY (ENABLE SPINUP) is transmitted by a SAS initiator port or expander port and is used to specify to a SAS target device that it may temporarily consume additional power (e.g., while spinning-up rotating media) while transitioning into the active or idle power condition state. The length of time the SAS target device consumes additional power and the amount of additional power is vendor specific. In general, the initial power surge is due to the crowbar current of the hard disk motor that spins the internal platter. This crowbar current can be as large as ten times the normal operating current used to maintain the platter spinning, move the read/write head and power the control electronics. For systems using arrays of hard disks, the spin-up of the platters are staged in time to prevent the crowbar currents of multiple hard drives from overwhelming the common power supply.
FIG. 1 illustrates one such known arrangement 100 within SAS topology including SAS blade servers 101 in communication with a high density tiered storage array 103 via a SAS blade switch 102. As noted, each storage array 103 is typically connected (e.g., by daisy chain) to another such array (not shown for clarity). Such storage array 103 is shown simplified for purposes of illustration and includes a SAS expander 106 and a power supply 105 in communication with multiple drives 104. The power supply 105 is sized to handle maximum spin up requirements and is therefore often oversized for normal operational requirements. The drives 104 may be SAS or SATA HDDs. In such an arrangement 100, spin up problems may exist with regard to disk drives 104 in which the given hard disk normally does not spin up its platter until the NOTIFY (ENABLE SPINUP) and the SCSI UNIT START/STOP command are both received from the SAS expander 106. If the SCSI START STOP UNIT (SBC-3) command is received, the command does not complete until NOTIFY (ENABLE SPINUP) is received or until the command timeouts. Thus, the NOTIFY (ENABLE SPINUP) is only a permission that gates spin-up, but does not actually start the spin-up process.
In general, SAS initiator devices and expander devices send NOTIFY (ENABLE SPINUP) while attached to SAS devices. The expander transmits one NOTIFY (ENABLE SPINUP) after power on when the enclosure is ready for initial spin-up. After the initial NOTIFY (ENABLE SPINUP), NOTIFY (ENABLE SPINUP) is transmitted periodically. Other than this, the selection of when and how often to transmit NOTIFY (ENABLE SPINUP) is outside the scope of the ANSI standard. Moreover, the SAS expander is not allowed to process the reception of a NOTIFY (ENABLE SPINUP) or propagate it. SAS expanders control the spin-up of SATA devices differently as SATA devices do not contain a power condition state machine similar to SAS. The typical method to prevent the spin-up of a SATA device is to not successfully complete speed negotiation of the physical link. That is, the speed negotiation does not progress past the COMWAKE event of the normal SATA speed negotiation sequence. In this state, the SATA drive is in the SATA HOLD state.
Normally, the spin-up of SATA disks is permitted by a host within the SAS topology by the SAS management protocol (SMP) target within the SAS expander. Specifically, a host uses the SAS expander SMP control to reset the serial link connected to the SATA drive to allow speed negotiation to complete. At this point, the SATA drive exits the SATA HOLD state and starts to spin-up the drive. For applications involving multiple hosts within a SAS topology or large number of arrays of hard disks, the coordination of spin-up within the topology is a difficult problem. The spin-up of large arrays of SATA hard drives within a SAS topology is a difficult problem with multiple hosts.
While the ANSI specification related to SAS teach the use of NOTIFY primitives for spin-up as part of the SAS Expander function to handle primitive broadcast, no teaching exists for using NOTIFY primitives to control SAS expanders. As described earlier, a SAS topology can contain a number of host, target, and expander devices. The expander is a multiple port device providing switching functionality among host, target, and other expander devices allowing any device to exchange information with any other device in the topology, though such topology is required to be without loops within each SAS domain.
There are no known techniques for SAS that address the problem of hard disk spin-up for an entire SAS topology. SAS expanders are expected to perform spin-up for the hard disks connected directly to that given SAS expander. For other protocols, there are some techniques for spin-up of hard disks that are different from the SAS expander model. Some methods are controlled using two electrical signals on the connector to the drive.
A first known method does not spin-up the platter until 3 seconds after power is applied to the drive. Staggered spin-up between hard drives is performed by staggering the power cycling of each drive. The 3 second delay allows the power supply to be stable before initiating the platter spin-up. This method may require additional components around the hard drive to control the power to the hard drive.
Another known method spins-up the platter when the host sends the SCSI START STOP UNIT command to the drive. This method may not work well in applications involving multiple hosts as the hosts need to coordinate the sending of this command in a fault tolerant manner.
A third known method waits 12 seconds times the modula 8 value of the numeric SEL_ID of the drive (as specified by the electrical connections). This allows 8 drives to spin-up in a coordinated manner over a period of 96 seconds. Unfortunately, this method may not work well for large arrays and may not be adaptive in the number of simultaneous drives allowed to spin-up.
Another known method is that the Seagate drive will not spin-up. That is, in this mode, the hard drive will not spin-up the platter until a different method is selected by the connector. This allows the hard disk spin-up to be controlled using a electrical interface on the disk. Again, some mechanism must be used to control this electrical interface to allow the staggered spin-up.
Some other approaches rely on idle timeouts to spin-down the disk and some protocol command to spin-up the disk, but do not describe mechanisms to stagger the spin-up of the hard disks.
As suggested above, there are commonly problems related to disk drive spin up where HDDs consume additional supply when they spin-up. The HDDs draw high current from a power supply for a few seconds: Ipeak After they are spun up, the active current draw is much smaller Iactive<Ipeak In a chassis with N HDDs (where N is an integer greater than 1), if you allow all HDDs to spin up simultaneously, the peak current is N*Ipeak Often, a system designer may stagger the time the HDDs spin up in group of n (where n is another integer greater than 1) whereby the peak current is (N−n)*Iactive+n*Ipeak<N*Ipeak Such staggered spin-up reduces the peak current to the power supply and thus reduces the cost of the power supply. However, a mechanism is required to control the spin up of each HDD individually, and no such mechanism adequately exists in regard to SAS topology.
It is, therefore, desirable to provide coordination of either multiple hosts or expanders in a fault tolerant manner allowing an entire SAS topology to spin-up SATA and/or SAS HDDs in a coordinated manner to reduce peak current draw. This would reduce peak current draw required and therefore reduce the power supply requirements and related costs.